Certain embodiments of the present invention relate to the field of compression and decompression of digital video signals, also referred to as coding, or encoding, and decoding. There is a substantial body of prior art in the area of digital video standards, encoding, and decoding, with a wide variety of implementations and data formats.
More specifically, certain embodiments of the present invention relate to the aspect of video encoding and decoding that is variously called “deblocking”, “loop filtering” or “post-processing”. The term “deblocking” is used herein.
Aspects of deblocking are addressed by the proposed MPEG-AVC/H.264 draft standard, document number JVT-G050, entitled “Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264 ISO/IEC 14496-10 AVC), which is currently being developed by the MPEG-ITU Joint Video Team. Embodiments of the present invention are an improvement of the system and method disclosed in application Ser. No. 10/246,977 filed on Sep. 19, 2002.
In deblocking of mixed field-frame pictures, there is a problem of how to specify the appropriate filter for use at the boundaries between vertically adjacent macroblocks (MBs) when one neighbor is field coded (field ordered) and the other is frame coded (frame ordered). If both MBs are frame coded, deblocking can be performed conventionally, as is well known in the art. If both MBs are field coded, there are various options. Deblocking is performed using conventional frame filtering methods, but operates on one field at a time.
In the case where the current macroblock (MB) is field coded and the neighboring MB that is vertically adjacent, above or below the current MB, is frame coded, there are various choices that may be made for specifying the filtering at the boundary between the two macroblocks.
In the AVC proposed standard, there is a structure called “super-macroblock” (super-MB) which comprises two vertically adjacent MBs. Super-MB is used when MB-level adaptive field-frame coding (MB-AFF) is enabled. When MB-AFF is enabled, each super-MB may be either field coded or frame coded. A super-MB that is frame coded comprises two vertically adjacent frame MBs (an upper frame MB and a lower frame MB). A super-MB that is field coded comprises two macroblocks, one from each field (a top field MB and a bottom field MB). That is, one of the MBs in a field coded super-MB covers a 16×16 array of pixels in the top field of the frame, and the other MB in the same field coded super-MB covers a 16×16 array of pixels in the bottom field of the same frame.
The FCD (Final Committee Draft) of the AVC (NT) standard specified the above aspect of the deblocking operation as:
“When mb_adaptive_frame_field_flag=1, a MB may be coded in frame or field decoding mode. For frame MB, deblocking is performed on the frame samples. In such a case, if neighbouring MB pairs are field MBs, they shall be converted into frame MB pairs (FIG. 8-3) before deblocking. For field MB, deblocking is performed on the field samples of the same field parity. In such a case, if neighbouring MB pairs are frame MBs, they shall be converted into field MB pairs (FIG. 8-3) before deblocking.”
The method described above has a number of problems. In the case of a frame MB, and where the neighboring MBs are field coded, the field coded MBs are “converted into frame MB pairs”. In other words, the lines from the two different fields are interleaved together as if they were from one frame which represents one instant in time, which in fact is incorrect. When lines of two different fields are interleaved and filtered together, the filtering tends to cause blurring between the fields. A reasonable assumption is that the encoder chose field coding for the neighboring MBs because the encoder determined that there is inter-field motion and, therefore, it is reasonable to expect inter-field blurring caused by the filtering.
When the current MB is field coded and the neighboring MBs are frame coded, they are “converted into field MB pairs” which causes another problem, since a 4×4 sub-block within a frame MB, when converted into a field MB structure, covers only 2 lines. As a result, a deblocking filter that has an extent of 3 or more pixels on each side of the boundary between MBs would require changing pixels beyond the boundary sub-block an extra time after the result of the normal filtering.
A more recent version of the AVC draft Standard, JVT-E146, referred as the DIS (Draft International Standard) replaces the FCD with the following:
“When mb_adaptive_frame_field_flag is equal to 1, all macroblock pairs are assumed to be reconstructed within the frame independent of the mb_field_decoding_flag.”
One interpretation of such a statement is as follows:
“All MBs are converted into frame order before deblocking, regardless of whether the MBs are field coded or frame coded.”
Such an interpretation is even more problematic than the version above in the FCD. In the DIS version, with all field coded MBs being combined into frame order and filtered in frame order, there is a large degree of inter-field blurring. When there is motion in the video there tends to be large differences between successive fields, particularly at the edges of objects, and constitutes one of the main reasons an encoder would choose field coding for MBs. Where there are large differences between successive fields, and the top and bottom fields located at one super-MB are combined into one frame, there are large differences between successive lines within the resulting combined MB, even if there are little or no differences between lines within any given field MB and even if there are no blocking artifacts in any one field. Such differences between lines, that are artificially induced by the conversion of field super-MBs into frame super-MBs, cause the deblocking filter to work to reduce those differences and cause blurring from one field to the next, thereby tending to defeat the advantage of choosing field coded super-MBs.
It is desirable to find a method of deblocking pictures of video that contain a mixture of field and frame coding such that blockiness is reduced and there are no additional image impairments introduced.
Further limitations and disadvantages of conventional, traditional, and proposed approaches will become apparent to one of skill in the art, through comparison of such systems with the present invention as set forth in the remainder of the present application with reference to the drawings.